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ABSTRACT 


This  research  was  initiated  due  to  a  report  claiming 
that  The  U.S.  Navy  significantly  overestimated  repair 
prices  in  a  Performance  Based  Logistics  (PBL)  reward  to  a 
contractor.  The  purpose  of  this  thesis  is  to  develop  a 
model  for  improving  the  prediction  of  repair  price  for  U.S. 
Navy  inventory  items . 

The  thesis  examines  several  prediction  methodologies, 
including  a  ratio-estimator  prediction  method  that  is  a 
modification  of  the  methodology  currently  in  use,  as  well 
as  regression  analysis.  In  contrast  to  the  ratio-estimator 
approach,  regression  is  able  to  utilize  a  wide  range  of 
predictor  variables,  several  of  which  are  evaluated  in  the 
thesis  research.  Results  of  this  analysis  reveal  that  a 
regression  model  with  logarithmic  transformations  yields 
more  accurate  predictions  of  repair  prices  than  the  current 
methodology.  This  improvement  is  seen  especially  for  items 
that  have  the  highest  replacement  price. 

One  feature  of  the  proposed  regression-based 
methodology  is  that  predicted  repair  prices  for  the  most 
expensive  items  are  substantially  lower  than  with  the 
current  methodology.  In  the  case  where  that  prompted  the 
thesis  research,  the  overstatement  of  benefit  from  the  PBL 
would  have  been  reduced  by  about  30  million  under  the 
proposed  methodology. 
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EXECUTIVE  SUMMARY 


The  U.S.  Navy  has  initiated  programs  to  team  with 
private  industry  for  logistics  support.  In  particular. 
Performance  Based  Logistics  (PBL)  initiatives  have  been 
undertaken  by  Naval  Inventory  Control  Point  (NAVICP) - 
Philadelphia  to  achieve  its  goals  for  improving  support  and 
lowering  total  ownership  cost  of  the  Naval  inventory  items 
under  its  management.  For  each  PBL  initiative,  NAVICP 
conducts  a  business  case  analysis  (BCA)  to  estimate  its 
cost  benefits  in  order  to  support  the  decision  to  award  the 
contract . 

This  research  was  initiated  as  a  result  of  a  report  by 
the  Department  of  Defense  Office  of  the  Inspector  General 
(DoD  IG)  claiming  that  NAVICP  overstated  cost  saving  in  the 
F/A-18E/F  Integrated  Readiness  Support  Teaming  (FIRST) 
Program  award  to  Boeing.  The  report  found  that  the  cost  of 
repairs  conducted  by  Boeing  were  substantially  lower  than 
what  the  Navy  paid  for  these  repairs,  based  on  its  repair- 
price  estimates.  Estimated  repair  prices  in  NAVICP' s  BCA 
analysis  were  more  than  two  hundred  percent  higher  than  the 
actual  repair  price  reported  by  Boeing.  Most  of  the 
discrepancy  was  attributed  to  repairable  items  that  had  the 
highest  replacement  price  ($10,000  and  above). 

The  thesis  research  examined  alternative  methodologies 
for  predicting  repair  prices  with  the  aim  of  improving  the 
prediction  accuracy  of  the  methodology  currently  used  by 
NAVICP.  The  current  methodology  uses  ratio  estimators  for 
a  categorization  of  repairable  items  based  on  their 
replacement  price.  Within  each  of  six  replacement  price 
categories,  the  average  of  the  ratios  of  repair  price  to 


xv 


replacement  price  is  calculated,  using  equal  weighting. 
For  an  item  with  no  historical  repair-price  data,  its 
repair  price  is  predicted  by  multiplying  the  appropriate 
ratio  by  its  replacement  price. 

A  modification  of  the  ratio  methodology  was 
considered,  which  weighted  the  averages  proportionally  to 
the  replacement  prices  of  items  within  a  category.  Analysis 
using  the  modified  ratio  methodology  offered  slightly 
improved  results  for  items  in  the  highest  replacement  price 
category.  However,  an  analysis  of  both  ratio  methods  found 
that  replacement  price  alone  is  not  a  strong  predictor  of 
repair  price. 

An  investigation  was  then  made  into  the  use  of 
regression  models  for  the  prediction  of  repair  prices. 
These  models  considered  variable  transformations  as  well  as 
additional  predictor  variables.  The  simplest  regression 
model  used  the  logarithm  of  replacement  price  to  predict 
the  logarithm  of  the  ratio  of  repair  price  to  replacement 
price.  More  extensive  regression  models  included 
categorical  predictor  variables  based  on  the  Federal  Stock 
Class  (FSC)  and  Local  Routing  Code  (LRC)  of  an  inventory 
item.  Stepwise  regression  procedures  were  used  to  identify 
models  that  had  good  predictive  power  without  overfitting. 

Regression  models  that  included  combined  categorical 
groupings  of  FSC  and  LRC  yielded  the  best  prediction  power. 
However,  the  level  of  improvement  was  small  when  compared 
to  the  simple  regression  model  that  used  only  the  logarithm 
of  replacement  price  as  a  predictor. 

The  prediction  methodologies  that  were  analyzed  were 
then  evaluated  on  the  repair  data  cited  in  DoD  IG' s  report 
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on  the  FIRST  program.  The  simplest  regression  model 
resulted  in  a  substantial  improvement  over  the  current 
methodology  for  the  inventory  items  that  were  considered. 
The  total  prediction  error  was  reduced  from  forty-seven 
million  dollars  (current  methodology)  to  fourteen  million 
dollars  (regression) . 

The  thesis  research  found  that  a  modification  of  the 
methodology  currently  used  by  NAVICP  to  predict  repair 
prices  can  improve  prediction  accuracy,  particularly  for 
items  with  the  highest  replacement  prices.  Because  repair 
prices  are  positively  related  to  replacement  prices, 
repairs  of  these  items  have  the  greatest  fiscal  impact. 
Improved  prediction  accuracy  can  be  achieved  by  adopting  a 
regression-based  methodology,  which  in  addition  to 
improving  prediction  accuracy  allows  for  the  inclusion  of 
multiple  predictor  variables.  Although  FSC  was  found  to 
moderately  improve  the  prediction  accuracy,  and  LRC  was 
found  not  to  greatly  improve  the  prediction  accuracy  of  the 
regression  models  considered,  only  a  limited  number  of 
potential  predictors  were  examined.  It  is  possible  that  a 
more  extensive  investigation  would  uncover  predictors  that 
substantially  improve  the  prediction  accuracy  of  these 
models . 
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A.  INTRODUCTION 


The  research  described  in  this  thesis  is  the  result  of 
an  initiative  by  Naval  Inventory  Control  Point  (NAVICP)- 
Philadelphia  to  replace  its  current  methodology  for 
predicting  the  repair  prices  of  its  inventory  items.  The 
focus  of  this  research  is  the  development  of  an  improved 
methodology  so  as  to  provide  a  more  accurate  prediction  of 
the  costs  of  repairing  Naval  inventory  items. 

B.  BACKROUND  OF  THIS  RESEARCH 

In  1999,  the  Navy  adopted  an  initiative  known  as  the 
Performance  Based  Logistics  (PBL)  that  transfers 
traditional  Department  of  Defense  (DoD)  inventory,  supply 
chain  and  technical  support  functions  to  commercial 
contractors  for  a  guaranteed  level  of  performance  at  the 
same  or,  ideally,  reduced  cost  [1] .  Under  the  PBL  program, 
the  contractor  is  responsible  for  a  range  of  activities 
including  warehousing,  transportation,  repair/overhaul , 
technology  insertion,  engineering  services,  guaranteed 
reliability,  and  warranty  management.  An  example  of  this 
concept  was  the  award  of  a  contract  to  The  Boeing  Company 
(hereafter  known  as  Boeing)  on  May  2001  for  a  pilot  program 
on  F/A-18E/F  Integrated  Readiness  Support  Teaming,  also 
known  as  the  FIRST  Program. 
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For  the  "FIRST"  PBL  initiative,  NAVICP  conducted  a 
Business  Case  Analysis1  (BCA)  based  on  a  five-year  period  to 
quantify  any  cost  benefits  that  the  Navy  might  realize  due 
to  this  collaboration.  These  cost  benefits  can  accrue  from 
either  cost  avoidance  or  cost  savings.  The  NAVICP  BCA, 
using  data  from  FY1995,  indicated  that  an  expected  saving 
of  $55.4  million  (later  adjusted  to  $52.4  million)  was 
achievable  over  a  five-year  period.  The  analysis  also 
anticipated  savings  of  $73.7  million  in  cost  avoidance 
relating  to  other  integrated  logistics  support  elements 
from  the  FIRST  Program  [1]  .  Thus  the  Navy  anticipated  a 
saving  of  $126.1  million  contracting  of  FIRST  program. 

However,  an  audit  conducted  by  the  Department  of 
Defense  Office  of  the  Inspector  General  (DoD  IG)  found  that 
the  business  case  that  the  Navy  used  to  justify  the  award 
of  the  FIRST  contract,  overstated  the  cost  of  DoD 
performance  by  $268.9  million.  Therefore,  instead  of  the 
Navy's  claim  of  saving  $126.1  million  over  the  first  five 
years,  the  analysis  showed  that  the  FIRST  program  actually 
cost  the  Navy  $142.8  million  more  than  the  traditional 
support  method  [1]  .  Hence  NAVICP  was  tasked  to  look  into 
the  accuracy  of  the  business  case  analysis  used  extensively 
to  develop  these  numbers.  Chapter  II  provides  a  more 
detail  description  of  the  "FIRST"  PBL  as  well  as  the  BCA 
used  in  this  program. 


1  BCA  is  a  process  designed  to  quantify  cost  area  such  as 
procurements,  repairable,  warehousing,  maintenance  cost  etc  used  to 
determine  Navy's  current  cost. 
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C.  OBJECTIVE  AND  SCOPE  OF  RESEARCH 

The  research  described  in  the  thesis  was  conducted 
under  the  sponsorship  of  the  Operations  Research  group  at 
NAVICP-Philadelphia .  The  aim  of  the  research  is  to  develop 
a  predictive  model  for  estimating  the  repair  prices  of 
components  when  the  procurement  price  is  known,  but  no 
actual  repair  data  are  yet  available. 


Inventory  items  are  classified  as  either  consumable  or 
repairable,  depending  on  whether  they  can  be  restored  to 
ready-f or-issue  condition  in  a  cost-effective  manner  when 
they  fail.  Accurate  prediction  of  the  cost  of  repairing  an 
item  is  an  important  step  in  determining  whether  the  item 
should  be  classified  as  consumable  or  repairable.  It  is 
also  important  to  the  effective  management  of  the  inventory 
system . 


Repair  prices  for  aviation  and  weapon  system  support 
can  be  difficult  to  predict.  For  example,  an  item  could 
be  repaired  in  one  of  several  types  of  facilities, 
including  Navy  depots,  inter-service  depots  or  commercial 
facilities,  thus  resulting  in  large  variations  in  repair 
prices.  The  ability  to  obtain  accurate  predictions  and  to 
quantify  uncertainty  in  these  predictions  is  important  to 
defense  organizations,  which  utilize  them  to  determine  the 
resources  needed  to  fully  optimize  and  fund  its  operations. 
Under-funding  could  result  in  a  delay  or  cancellation  of 
equipment  replacement,  whereas  over-funding  could  deny 
resources  to  other  operations. 
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In  August  2003,  the  NAVICP  Operations  Research  group 
was  tasked  to  develop  a  methodology  for  improving 
prediction  accuracy  for  repair  prices,  in  light  of  the 
over-prediction  of  savings  for  the  FIRST  program.  Because 
data  on  repair  prices  were  not  always  available  when 
conducting  BCA  for  the  FIRST  program,  NAVICP  established 
the  repair  prices  for  newly  provisioned  items  using  the 
repair  price  matrix  for  estimating  the  repair  prices  versus 
historical  data  for  similar  items  available  at  the  Naval 
depots  [1]  .  The  repair  matrix  is  based  on  a  stratification 
of  repairable  items  (mainly  aviation  and  weapon  system 
components)  by  replacement  price  into  six  different  price 
categories.  The  ratio  of  the  repair  to  replacement  price 
is  estimated  within  each  category.  Repair  prices  for  newly 
provisioned  items  are  determined  by  multiplying  the 
replacement  price  with  the  appropriate  ratio  of  repair  to 
replacement  prices.  Chapter  II  discusses  the  repair  matrix 
in  detail,  which  was  used  as  a  baseline  for  comparison  in 
the  thesis  research. 

D .  METHODOLOGY 


To  facilitate  the  thesis  research,  NAVICP  provided 
data  for  1,420  newly  provisioned  aviation  and  weapon  system 
inventory  items.  Information  of  these  inventory  items 
include  nomenclatures,  federal  stock  class,  local  routing 
codes,  replacement  prices  and  repair  prices.  Other  related 
repair  processes  and  pricing  information  were  subsequently 
obtained  from  the  Material  Budgeting  and  Industrial  Support 
Departments  at  NAVICP. 
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This  thesis  reviews  the  appropriateness  of  various 
statistical  analysis  methodologies,  in  an  attempt  to 
improve  the  accuracy  of  repair-price  predictions.  The 
ratio-estimator  model  used  by  NAVICP  does  not  allow  for  the 
use  of  other  potential  explanatory  variables  that  may  aid 
in  predicting  repair  prices.  Other  methods,  such  as 
regression,  can  make  use  of  a  range  of  explanatory 
variables.  This  thesis  describes  statistical  analyses  to 
evaluate  the  benefits  of  including  explanatory  variables  in 
addition  to  replacement  price.  The  results  of  these 
analyses  are  then  compared  to  predictive  accuracy  of  the 
current  methodology.  All  data  analyses  are  performed  using 
the  statistical  software  package  S-Plus®. 

E.  BACKGROUND  OF  NAVICP 


The  objective  of  this  section  is  to  provide  a  brief 
understanding  of  the  operations  of  NAVICP.  The  mission  of 
NAVICP  is  to  provide  program  and  support  for  the  weapon 
systems  that  will  keep  its  Naval  forces  mission-ready  [7]  . 
This  mission  is  carried  out  by  a  single  command 
organization  operating  as  a  tenant  activity  of  Naval 
Support  Activities  in  Mechanicsburg  and  Philadelphia. 

NAVICP  was  established  in  1995,  following  the  merging 
of  Ships  Parts  Control  Center  (SPCC)  in  Mechanicsburg  and 
Aviation  Supply  Office  (ASO)  in  Philadelphia.  This  merger 
allowed  the  Navy  to  bring  together  all  inventory  support 
functions  under  a  single  command,  ensuring  a  timely 
delivery  of  quality  program  and  logistics  support  to  keep 
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its  Naval  forces  mission-ready  [7]  .  The  merger  is  also 
part  of  the  Navy's  effort  to  reduce  cost  and  infrastructure 
activities  as  well  as  to  standardize  inventory  management 
procedures . 


The  primary  activity  for  the  Philadelphia  site  is 
aviation  and  weapon  system  support.  Among  the  aircraft 
supported  are  the  F/A-18  and  the  V-22  as  well  as  various 
engines,  common  avionics,  and  support  equipment.  In 
contrast,  their  Mechanicsburg  counterpart  handles  support 
for  hull,  electrical,  mechanical,  and  electronic  components 
and  repair  parts  for  ships,  submarines  and  weapon  systems. 


This  section  discusses  the  departmental  activities  of 
three  main  offices  that  are  closely  related  to  the  accuracy 
of  the  repair  price,  namely  Material  Budgeting  Department 
of  Comptroller  Directorate  (P-code  013) ,  Industrial  Support 
Department  of  Operations  Directorate  (P-code  034)  and 
Operations  Research  Department  of  Planning  and  Data  Systems 
Directorate  (P-code  0412)  .  Figure  1  gives  an  overview  of 
the  NAVICP  organization. 
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Figure  1 .  NAVICP  Philadelphia  Organization  Chart  [After 
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The  Comptroller  Directorate  (P-code  01)  is  responsible 
for  controlling  the  development,  preparation  and 
justification  of  all  NAVICP  budgetary  requirements.  Its 
Material  Budget  department  administers  and  controls  the 
financial  aspects  of  inventory  management,  develops  and 
justifies  initial  replenishment  as  well  as  repair  budgetary 
requirements  for  Naval  ship  material  support  [4]  .  They  are 
the  primary  users  of  the  repair  price  matrix  used  to 
determine  the  final  net  price  (including  inflation, 
overheads,  burdening,  etc.)  charges  to  the  customer. 


The  primary  functions  of  the  Operations  Directorate 
(P-code  03)  involve  providing  supply  policies,  customer 
liaison,  technical  support  and  forecasting/planning  of  the 
Navy  Aviation  Operations.  The  Industrial  Support 
Department  is  responsible  for  the  execution  and  management 
of  the  Aviation  Depot  Level  Repairable  (AVDLR)  repair 
program.  The  team  coordinates  component  repair  work 
executed  by  the  Naval  Aviation  depots  (NADEPs) ,  and  the 
Inter-Service  component  repair  work  executed  by  the  Army 
Depots  and  Air  Force  Logistics  Centers  [8] .  It  obtains  and 
compiles  the  repair  pricing  (from  organic  depot, 
DMISA/NAPRA  and  Commercial  contractors)  required  for 
computation  of  budgetary  and  contracting  requirements. 
Additionally,  the  team  supports  NAVAIR  Scheduled  Depot 
Level  Maintenance  (SDLM)  functions  and  the  Management 
Control  Activity  (MCA)  for  Government  Furnished  Material 
(GFM)  contracts,  and  serves  as  the  focal  point  for 
information  and  problem  resolution  associated  with  depot 
level  repairs. 


The  Planning  and  Data  Supply  Directorate,  P04, 
analyzes,  develops  and  recommends  plans,  policies  and 
procedures  to  improve  practices  and  activities  at  NAVICP. 
The  Operations  Research  group  is  responsible  for  developing 
the  repair  price  matrix  for  estimation  of  repair  prices. 
Figure  2  shows  the  activities  of  each  department  relating 
to  repair  prices. 

Figure  2 .  Responsibilities  of  each  department  relating  to  repair 

prices . 


P0412  uses  the  data 
provided  by  P034  to 
develop  the  repair 
price  matrix 


Customers 

P013  uses  the  repair 

(Fleet ) 

<  n 

matrix  to  determine  net 

prices  for  customers 

F.  THESIS  ORGANIZATION 


This  thesis  is  divided  into  five  chapters. 


Chapter  I  gives  an  introduction  to  NAVICP  and  its 
organization  responsibilities.  It  also  describes  the 
background  for  this  thesis  research,  the  objective  and  the 
methodologies  used  to  conduct  the  research. 
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Chapter  II  discusses  in  detail  NAVICP  repair  processes 
and  its  pricing  methodologies  and  Business  Case  Analysis 
(BCA)  used  for  Performance  Based  Logistics  (PBL) . 


Chapter  III  describes  the  sources  of  data  used, 
including  filtering  of  unwanted  information  from  the  data. 
This  chapter  also  looks  into  the  current  ratio  methodology 
adopted  by  NAVICP  as  well  as  proposing  an  alternative  ratio 
method  that  could  help  improve  the  prediction  power  of  the 
repair  matrix. 


Chapter  IV  presents  the  results  of  the  analysis  and 
comparisons  to  the  predictive  accuracy  obtained  with  the 
current  pricing  matrix. 

Chapter  V  summarizes  the  research  findings  and 
presents  recommendations  for  future  research. 
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II.  OVERVIEW  OF  NAVY  REPAIR  PROCESS 


A.  INTRODUCTION 


This  chapter  describes  in  detail  the  U.S.  Navy's 
repair  processes  (depot-level,  commercial  or  inter-service) 
and  its  importance  to  budgeting  and  contracting.  These 
processes  were  introduced  in  Chapter  I  in  relation  to  the 
Performance  Based  Logistics  effort  for  the  "FIRST"  program, 
which  led  to  the  investigation  of  the  accuracy  of  its 
current  methodology  used  in  its  Business  Case  Analysis  for 
predicting  repair  prices. 

B.  OVERVIEW  OF  NAVICP  REPAIR  PROCESS 

Repair  price  refers  to  the  actual  unit  price  that 
NAVICP  pays  to  repair  an  item.  The  repair  may  be  performed 
by  any  one  or  combination  of  the  following; 

1 .  Commercial _ facilities _ (def initized _ contract)  . 

Commercial  repair  information  can  be  obtained  from  the 
Navy's  contract  status  file.  This  file  states  the 
level  of  effort  to  inspect,  repair  and  modify  an  item 
for  that  particular  program. 

2.  Organic  (Navy)  depots  (NADEPS) .  Repair  performed  at 

one  of  the  three  Naval  repair  facilities  located  in 
California,  Florida  and  North  Carolina.  Repair 

pricing  consists  of  actual  material  cost,  labor  cost 
and  Navy  Industrial  Fund  (NIF)  rate2. 

2  The  NIF  rate  is  established  as  a  cost  recovery  rate  for  labor  cost, 
overhead  costs,  transportation,  etc.  and  may  vary  for  different 
Designated  Overhaul  Points  (DOP) . 
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3.  Depot  Maintenance  Inter-Service  Agreement  (DMISA) . 


Repairs  performed  through  an  inter-service  agreement 
with  the  Army  or  the  Air  Force. 


The  projected  repair  price  for  each  repairable  item, 
which  is  stored  in  data  element  B055A,  is  usually  updated 
annually  and  does  not  actually  display  the  "latest"  price. 
NAVSUP  directives  concerning  repair  price  updates  for 
repairable  items  dictate  that  it  recover  any  losses  over  a 
period  of  approximately  twelve-fifteen  months.  For  a  given 
item,  the  repair  price  in  B055A  represents  a  computed 
weighted  average  repair  price  for  any  or  all  members  of  the 
family,  and  from  all  sources.  Thus,  each  inventory  item 
classified  as  either  a  head  or  member  of  the  family, 
depending  on  the  importance  of  the  components  in  its 
family,  will  have  the  same  computed  weighted  average  repair 
price . 


Repair  prices  are  determined  by  computing  weighted 
averages  of  prices,  with  the  weights  proportional  to  the 
number  of  items  repaired  at  each  price  category.  For 
example,  there  may  be  commercial  orders  and  organic 
repairs,  for  a  particular  item,  from  two  Navy  depots 
(NADEP)  that  contribute  to  an  average  price  of  $14,190  as 
shown  in  Table  1.  Thus,  the  fleet  could  pay  $14,190  even 
though  the  repair  price  may  cost  $22,500. 
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Table  1 .  Computed  Weighted  Average  Pricing  Methodology 


NUN 

Nomenclature 

Source 

Qty 

Price 

Total 

011199660 

Amp 1 i f i e  r 

Organic  1 

2 

$22,500 

$45, 000 

Radio 

Organic  2 

6 

$9,  800 

$58, 800 

Commercial 

12 

$15, 000 

$180,000 

Total 

20 

$283,800 

Average 

$14,190 

New  Repair 

Price 

$14 

i,190 

NUN  is  the  identification  number  of  the  inventory  item.  Source  refers 
to  where  the  repair  was  performed. 


Every  year,  the  Industrial  Support  Department  provides 
the  list  of  organic  as  well  as  DMISA  repair  prices.  These 
updated  repair  prices  are  then  used  by  the  NAVICP 
Operations  Research  group  to  generate  the  repair-price 
matrix  to  predict  repair  prices  for  newly  provisioned 
items.  The  material  budgeting  department  then  uses  these 
prices  to  determine  the  standard  and  net  prices3,  which 
include  cost  recovery  for  transportation,  depot  washout, 
obsolescence,  testing,  taxes  etc.,  as  illustrated  above  in 
Figure  2 . 


3  Standard  Price  is  the  price  the  customer  pays  for  a  new  issue  for  a 
consumable  or  Aviation  Depot  Level  Repairable  (AVDLR)  item.  Net  Price 
is  the  price  the  customer  pays  for  an  AVDLR  with  carcass  turn-in.  These 
prices  are  determined  by  multiplying  a  cost  recovery  rate  to  the  repair 
prices . 
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C.  PERFORMANCE  BASED  LOGISTICS  FOR  F/A-18E/F  AIRCRAFT 

Under  the  PBL  program,  NAVICP  awards  a  contract  to  a 
single  supplier.  This  supplier  then  provides  the  material 
directly  to  the  fleet  in  time  to  meet  the  customer' s 
requirement.  This  is  achieved  without  the  intervention  of, 
or  need  for,  government  inventory  managers  or  intervening 
storage  and  material  handling  systems  while  providing 
increased  product  reliability  and  reducing  total  cost  to 
the  Fleet  Customer  and  the  Navy  [6] .  One  of  first 
contracts  issued  under  the  PBL  program  on  May  2001  was  for 
the  F/A-18E/F  Integrated  Readiness  Support  Team  program. 
NAVICP  awarded  Boeing  a  five-year  contract  with  an  award- 
fee  provision  based  on  performance  requirements.  The 
contract  covered  procurement  of  initial  and  replenishment 
spares  for  519  repairable  parts  and  5856  consumable  parts 
as  well  as  repair  of  the  repairable  parts  [1] . 

For  each  PBL  initiative,  NAVICP  and  NAVAIR  conduct  a 
Business  Case  Analysis  (BCA)  to  justify  the  amount  of  cost 
savings  or  cost  avoidance4  benefits  through  the  contract. 
The  BCA  process  involves  determining  the  Navy' s  current 
cost  of  doing  business.  For  the  FIRST  Program,  the 
"without  FIRST"  cost  is  compared  with  the  cost  to  the  Navy 
under  a  PBL  arrangement.  The  "with  FIRST"  cost  includes 
both  the  PBL  supplier' s  costs  as  well  as  residual  costs 
that  the  Navy  will  retain  even  under  a  PBL  arrangement. 
Some  cost  areas  considered  in  the  BCA  are  depot  repair, 
wholesale  and  retail  spare  parts  procurements,  warehousing, 

4  Cost  avoidance  is  calculated  based  on  overheads,  operations  and 
labor  saving  by  NAVICP  due  to  logistic  support. 
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transportation,  fleet  maintenance  labor,  fleet  consumables 


sustaining  engineering,  NAVICP  operating  costs,  PBL 
administrative  costs  and  other  miscellaneous  costs  [5]. 


A  BCA  conducted  by  NAVICP  indicated  that  an  expected 
savings  of  $55.4  million  (later  adjusted  to  $52.4  million) 
was  likely  over  a  5-year  period  from  the  FIRST  program. 
The  analysis  also  found  that  savings  of  $73.7  million  in 
cost  avoidance  relating  to  other  integrated  logistics 
support  elements  from  the  FIRST  program  was  achievable. 
Table  2  summarizes  the  Navy's  reported  five-year  $126.1 
million  cost  avoidance  relating  to  the  FIRST  contract  with 
Boeing  [ 1 ] . 


Table  2.  FIRST  Program  Savings  (1999)  -  Without  FIRST  and  With 

FIRST  (in  millions  of  dollars)  [1] . 


Description 

Without 

FIRST 

With 

FIRST 

Cost  Increase/ 
(Cost  Avoidance) 

NAVICP  BCA 

Material  costs 

$779.00 

$771 . 50 

$(7.50) 

Operations  cost 

$108 .10 

$63.20 

$ (44 . 90) 

Subtotal  NWCF  cost 

$887 .10 

$834 .70 

$ (52 .40) 

NAVA I R  BCA 

Non-supply  support 
elements 

$1,531.20 

$1,457 .50 

$ (73.70) 

Total 

$2,418.30 

$2,292.20 

$126.10 

Non-supply  support  elements  refer  to  cost  avoidance  due  to  engineering, 
integrated  logistic  support,  information  systems  and  supporting 
equipment.  Cost  avoidance  is  calculated  based  on  overheads,  operations 
and  labor  saving  by  NAVICP  due  to  logistic  support . 
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However,  DoD  IG  found  that  these  claimed  savings  were 
overstated,  and  that  the  FIRST  program  actually  resulted  in 
a  cost  increase  of  $142.8  million.  Table  3  summarizes  the 
adjustments  made  to  the  BCA  [1]  .  The  huge  difference  in 
the  estimation  of  cost  savings  led  to  an  investigation.  It 
appears  that  NAVICP  used  an  outdated  repair  price  matrix  to 
perform  the  business  case  analysis  for  determining  repair 
price  of  an  item  with  little  or  no  historical  cost  data 
(i.e.  newly  provisioned  items)  [1]. 


Table  3.  DoD  IG  Corrected  FIRST  Program  Savings  [1] . 


Description 

IG 

Corrected 

IG  Corrected 

Cost 

Increase/ 

Without  FIRST 

With  FIRST 

(Cost 

Avoidance) 

NAVICP  BCA 

Material  costs 

$ 

573 . 80 

$  783.10 

$ 

209.30 

Operations  cost 

$ 

119.50 

$  63.20 

$ 

(56.30) 

Subtotal  NWCF  cost 

$ 

693.30 

$  846.30 

$ 

153.00 

NAVA I R  BCA 

Non-supply  support 
elements 

$ 

1, 531.20 

$  1,521.00 

$ 

(10.20) 

Total 

$ 

2,224.50 

$  2,367.20 

$ 

142.80 

Non-supply  support  elements  refer  to  cost  avoidance  due  to  engineering, 
integrated  logistic  support,  information  systems  and  supporting 
equipment.  Cost  avoidance  is  calculated  based  on  overheads,  operations 
and  labor  saving  by  NAVICP  due  to  logistic  support . 
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D.  THE  NAVI CP  METHODOLOGY  FOR  PREDICTION  OF  REPAIR  PRICES 

This  section  describes  how  NAVICP  uses  actual  repair 
prices  to  generate  the  repair  price  matrix,  and  the 
importance  of  these  costs  to  the  accuracy  of  this  matrix. 


NAVICP  developed  the  repair  price  matrix  in  1986, 
based  on  a  study  of  actual  repair  prices  of  items  across 
all  weapon  systems  and  aviation  items  that  had  procurements 
in  the  previous  two  years.  The  repair  matrix  groups 


inventory 

items  based  on 

replacement 

price  into 

six 

categories . 

Within  each 

category. 

repair  price 

is 

predicted 

by  applying  a 

common  multiplier  to 

the 

replacement  price.  The  multiplier  is  calculated  by  taking 
the  average  of  the  ratios  of  repair  to  replacement  price. 
For  a  newly  provisioned  item  (that  has  not  generated  a 
repair  history) ,  its  repair  price  is  predicted  by 
multiplying  its  replacement  price  by  the  appropriate 
multiplier.  For  example,  the  multiplier  for  items  with 
replacement  prices  less  than  $999  (Category  1)  is  48%,  and 
an  item  with  a  replacement  price  of  $800  has  a  predicted 
repair  price  of  $384.00  (48%  of  $800)  .  This  repair  price 

matrix  used  by  NAVICP  in  justifying  cost  saving  from  the 
FIRST  program  is  shown  in  Table  4. 
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Table  4.  Repair  Price  Matrix  used  for  FIRST  Program  in  1995 

[After,  1] . 


Replacement  Price  of 
Inventory  Item  (dollars) 

Predicted  Repair  Price,  as 
Percentage  of  Replacement  Price 

1  -  999 

48 

1,000  -  2, 999 

32 

3, 000  -  9, 999 

30 

10, 000  -  24, 999 

24 

25, 000  -  49, 999 

20 

50,000  + 

15 

The  percentage  in  the  second  column  is  the  average  of  the  ratios  of 
repair  to  replacement  price  across  all  items  in  the  corresponding 
category. 
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III.  OVERVIEW  OF  CURRENT  METHODOLOGY 

A.  INTRODUCTION 

This  chapter  describes  the  source  of  data  used  in  the 
analysis.  It  also  describes  the  current  prediction 
methodology  adopted  by  NAVICP,  and  it  proposes  an 
alternative  method  that  could  help  improve  prediction 
accuracy . 

B.  DATA  USED  IN  THIS  RESEARCH 

The  data  used  for  the  cost  estimation  are  obtained 
from  the  Master  Data  File  from  NAVICP' s  Oracle  server. 
This  data  file  is  widely  used  in  NAVICP  for  tracking 
inventories,  quarterly  demands,  replacement  and  repair 
prices  etc.  The  data  also  include  other  variables  that  may 
be  useful  in  predicting  repair  prices.  An  excerpt  of  the 
data  used  for  the  analysis  in  this  thesis  is  shown  in  Table 
5  . 
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Table  5.  An  Excerpt  of  the  Data  Used  in  the  Thesis  Research. 


COG 

FSC 

NUN 

NOMENCLATURE 

REPL_ 

PRICE 

RPR_ 

PRC 

FRC 

FGC 

LRC 

7R 

5930 

000083636 

SWITCH  ASSEMBLY 

4242 

1081 

H 

W6AB 

QAC 

7R 

5985 

000014545 

SWITCH 

10412 

1296 

H 

DMF 

7R 

6110 

000016632 

REGULR, VOLTAGE 

19021 

648 

H 

DTQ 

7R 

1620 

000049840 

PISTON  ASSEMBLY 

31093 

9179 

H 

MXD 

1R 

2840 

998149318 

RING,  COMPRESSOR 

6805 

0 

H 

SVA 

7R 

1560 

008666688 

TANK, FUEL, 
AIRCRAFT 

20210 

3998 

H 

LHF 

OR 

6130 

014663481 

POWER  SUPPLY 

695 

144 

H 

T16 

7R 

5998 

012019353 

CIR . CARD  ASSY 

5352 

739 

H 

Q3C 

COG  is  the  Cognizance  Codes  which  is  used  to  identify  if  item  is 
repairable  or  consumable .  FSC  is  the  Federal  Stock  Class  assigned  to 
each  item  to  classify  them  into  different  categories.  NUN  is  the 
National  Item  Identification  number  used  to  identify  an  approved 
inventory  item.  FRC  is  the  Family  Relationship  Code  which  identifies 
if  an  item  is  the  head  or  member  of  the  family.  FGC  is  the  Family 
Group  Code  used  to  identify  an  item  within  a  family.  LRC  is  the  Local 
Routing  Code  used  to  determine  the  internal  organizational  component 
(by  Platform)  to  which  item  inputs  are  to  be  routed  for  action. 


The  first  three  fields  (COG,  FSC  and  NUN)  give  the 
identity  of  the  supply  item.  COG  is  the  Cognizance  Code, 
which  is  used  to  identify  an  item  as  repairable  or 
consumable.  Items  with  COG  code  'OR'  are  funded  and 
managed  through  special  appropriations.  Code  '1R'  refers  to 
items  that  are  non-repairable,  also  known  as  consumables. 
Code  '7R'  refers  to  items  that  are  repairable. 


Federal  Stock  Class  (FSC)  is  a  four  digit  code 
assigned  to  each  item  to  classify  them  into  different 

categories,  such  as  aircraft  fuel  pumps,  compressors  or 

circuit  boards.  Appendix  A  shows  a  full  listing  of  FSC 

codes  used  in  this  analysis.  The  national  item 


20 


identification  number  (NUN)  is  an  eight-digit  code  used  to 


uniquely  identify  an  inventory  item.  The  family 

relationship  code  (FRC)  identifies  an  item  as  either  the 
head  or  member  of  its  family.  Each  family  of  items  is 


identified 

by  a 

family  group  code 

(FGC) 

that 

is  blank  if 

the  family 

consists 

of  a 

single 

item 

(also 

known  as  a 

bachelor  item) . 

The 

local 

routing 

code 

(LRC) 

is  a  three- 

letter  code  used  to  determine  the  internal  organizational 
component  to  which  item  inputs  are  to  be  routed  for  action. 
The  first  letter  of  the  code  identifies  the  platform  group 
that  manages  the  items.  Appendix  A  shows  a  full  listing  of 
LRC  codes  used  in  this  analysis. 


The  data  set  includes  a  total  of  1,420  newly 
provisioned  inventory  items  available  from  January  to  June 
2003.  Not  all  the  information  in  the  data  set  provided  by 
NAVICP  is  appropriate  for  this  analysis.  Table  6  shows  a 
list  of  items  to  be  excluded  from  the  analysis. 


Table  6.  Filtering  of  data  set  to  be  used  for  this  analysis 


Total  number  of  Inventory  Items 

1420 

Less 

1.  Items  with  COG  =  'OR'  or  '1R' 

21 

2.  Items  with  lack  of  data  or  Repair  Price  <  0 

6 

3.  Items  that  are  not  head  of  the  family 

50 

Total  number  of  Items  deemed  useful  for  this  analysis 

1343 
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Items  with  COG  code  'OR'  are  funded  and  managed 
through  special  appropriations  whilst  items  with  Code  'lR' 
are  categorized  as  non-repairable,  also  known  as 
consumables.  Hence,  predicting  the  repair  prices  for  these 
twenty  one  items  would  not  be  of  fair  representation  or 
accurate.  Only  items  with  Code  7R,  items  that  are 
repairable,  are  considered  in  this  analysis. 


There  are  four  inventory  items  that  have  no 
information  other  than  the  inventory  identification  number. 
Also,  there  are  two  items  with  no  repair  price  information 
and,  therefore,  these  data  are  removed  from  the  data  set. 


NAVI CP  groups  repair  price  data  by  families  of  items. 
Each  item  within  a  family  has  the  same  estimated  repair 
price.  To  avoid  using  redundant  information,  only  the  data 
reported  for  the  head  of  the  family  are  used.  Fifty  items 
corresponding  to  non-head  members  of  families  are  removed 
from  the  analysis. 
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c. 


CURRENT  PREDICTION  METHODOLOGY 


The  current  methodology  adopted  by  NAVICP  is  to 
determine  the  average  of  the  ratios  of  repair  to 
replacement  price  across  all  inventory  items  within  a 
specified  category.  There  are  a  total  of  six  different 
categories  based  on  replacement  price.  The  following  shows 
the  calculation  used  to  determine  the  average  cost  ratio 
for  category  j  : 


A =2 X 


i= 1 


A 

x„ 


where  Y..  and  X;j  are  the  repair  and  replacement  prices, 
respectively,  for  item  i  in  category  j  .  In  this  notation  a 
weight  denoted  Wtj  is  also  used  to  calculate  the  average. 
NAVICP  uses  equal  weights,  Wij=\lnj,  to  calculate  the 
average  ratio,  where  nj  is  the  number  of  items  in  category 
j  .  The  predicted  repair  price  for  an  item  having  no 
historical  repair  price  data,  with  replacement  price  X 
belonging  to  category  j ,  is  then  given  by  Y =  PjX  • 


Based  on  the  data  provided  by  NAVICP  (total  of  1,420 
inventory  items),  the  current  repair  matrix  is  shown  in 
Table  7.  This  table  also  presents  a  summary  of  the 
variability  and  correlation  between  repair  and  replacement 
prices  for  these  items.  From  Table  7,  it  is  observed  that 
there  is  moderate  correlation  between  replacement  and 
repair  prices  for  majority  of  the  price  categories,  which 
indicates  that  a  strong  linear  relationship  between  the  two 
variables  is  unlikely  to  exist.  This  implies  that  using 
replacement  price  as  the  sole  variable  to  predict  the 
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repair  price  will  leave  unexplained  much  of  the  variability 
in  repair  prices. 


Table  7 .  Current  Repair  Price  Matrix 


Price  Category 

Number 

of  Items 

Correlation 
between  repair  and 
replacement  price 

Predicted  Repair 
Cost,  as 
Percentage  of 
Replacement  Price 

UP  TO  $999 

33 

0.561 

60 . 91 

$1,000  TO  $2,999 

188 

0.269 

42 . 52 

$3,  000  TO  $9, 999 

457 

0.238 

34 . 01 

$10, 000  TO  $24, 999 

354 

0.282 

25 . 14 

$25,000  TO  $49, 999 

155 

0.281 

18.44 

$50,000  AND  ABOVE 

156 

0.283 

16.00 

Overall 

1343 

0 .447 

- 

The  percentage  in  the  last  column  is  the  average  of  the  ratios  of 
repair  to  replacement  price  across  all  items  in  the  corresponding 
category. 


D.  MODIFIED  RATIO  METHODOLOGY 

An  alternative  ratio  method  is  considered  where  the 
weights  W-  are  chosen  to  be  proportional  to  Xtj  .  These 
ratio  estimators  take  the  following  form: 


where  Yj  and  Xj  are  the  average  repair  and  replacement 
prices,  respectively,  for  all  items  in  category  j  .  Items 
with  higher  replacement  prices  in  a  category  are  assigned 
more  weight  using  the  modified  ratio  method  than  with  the 
method  currently  used  by  NAVICP.  Results  of  applying  the 
modified  ratio  method  to  the  1,343  items  used  in  the 
analysis  are  shown  in  Table  8. 


Table  8 .  Repair  Price  Matrix  (Modified  Ratio  Method) 


Price  Category 

Number 

of  Items 

Predicted  Repair 
Price,  as  Percentage 
of  Replacement  Price 

UP  TO  $999 

33 

59.36 

$1,000  TO  $2,999 

188 

41 . 68 

$3,000  TO  $9,999 

457 

32 .56 

$10, 000  TO  $24, 999 

354 

24 . 90 

$25, 000  TO  $49, 999 

155 

18 . 64 

$50,000  AND  ABOVE 

156 

12 .40 

Overall 

1343 

- 

The  percentage  in  the  last  column  is  the  average  of  the  ratios  of 
repair  to  replacement  price  across  all  items  in  the  corresponding 
category. 
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E. 


A  COMPARISON  OF  RATIO  PREDICTION  METHODOLOGIES 


A  box  plot  provides  an  excellent  way  to  visualize  many 
aspects  of  a  distribution  in  a  data  set,  particularly 
determining  the  median  and  variation  changes  between 
different  categories.  Figure  3  and  Figure  4  show  the 
median,  variability  and  outliers  of  the  relative  prediction 
error  (between  repair  and  predicted  repair  prices)  using 
the  current  and  modified  ratio  methodology  respectively. 
Relative  prediction  error  is  defined  as  follows: 


M) 


It  should  be  noted  that  data  points  with  relative 
prediction  error  greater  than  0.7  are  eliminated  to  enhance 
graphical  display. 


It  is  observed  that  the  spread  increases  with  each 
price  categories  and  large  error  can  be  seen  in  the  region 
of  the  higher  priced  categories  for  both  plots.  For  both 
methodologies,  the  median  and  variations  are  reasonably 
similar  for  price  categories  below  $50,000.  Slight 
improvement  in  the  prediction  error  can  be  seen  in  the  last 
categories  for  inventory  items  with  replacement  prices  of 
$50,000  and  above. 
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A  total  of  37  out  of  1343 
prediction  error  greater  t 
enhance  graphical  display. 


plots  of  Relative  Pre< 
for  the  Modif: 


Up  to  $999  $1,000-$2,999  $3,000-! 


3 

t 

y- 


A  total  of  33  out  of  134! 
prediction  error  greater 
enhance  graphical  displa; 


In  Table  9,  it  is  rather  evident  that  both  methods 
show  little  success  when  predicting  repair  prices  for 
inventory  items  with  replacement  prices  greater  than 
$10,  000.  Again,  the  largest  error  can  be  seen  in  the  last 
category,  whereby  75  percent  of  the  observations  have 
approximately  245  percent  prediction  errors  for  the  current 
ratio  method.  The  modified  ratio  method  shows  an 
improvement,  but  still,  the  relative  prediction  errors  are 
greater  than  or  equal  to  160  percent  for  almost  25  percent 
of  items  in  this  category. 


Both  ratio  methods  indicate  that  replacement  price 
alone  may  be  a  poor  predictor  of  the  repair  price,  and  that 
the  relationship  between  the  two  analyzed  variables  may  be 
non-linear.  Chapter  IV  investigates  the  use  of  a 
logarithmic  transformation  in  the  regression  method,  as 
well  as  including  categorical  variables,  that  may  help  to 
improve  the  prediction  of  repair  prices. 
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Table  9.  Quantiles  of  Absolute  Relative  Prediction  Error  (Percentages)  of  current  and 

modified  ratio  method. 


Category 

Sample 

Size 

Current  Ratio  Method 

Modified  Ratio  Method 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

up  to  $999 

33 

16.47 

28.96 

38.66 

32.59 

16.48 

25.95 

46.35 

36.31 

$1,000  -  $2,999 

188 

21.37 

36.94 

75.59 

71.28 

20.91 

37.07 

73.92 

70.39 

$3,000  -  $9,999 

457 

22.70 

47.12 

106.44 

96.90 

22.08 

45.62 

99.72 

93.29 

$10,000  -  $24, 999 

354 

23.33 

52.63 

140.33 

131.67 

23.42 

52.52 

139.34 

131.04 

$25, 999  -  $49, 999 

155 

30.74 

61.38 

146.42 

170.38 

30.22 

62.14 

141.60 

166.56 

$50,000  and  above 

156 

40.26 

78.18 

245.32 

320.73 

37.21 

65.14 

160.36 

234.88 

INJ 
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IV .  METHODOLOGY 


A.  INTRODUCTION 


In  this  chapter  attention  is  focused  on  the  use  of 
regression  models  to  predict  the  repair  price  of  an 
inventory  item  from  its  replacement  price  and  possibly 
other  explanatory  variables.  This  chapter  will  look  at 
regression  analysis  with  logarithmic  terms  as  well  as 
categorical  variables  such  as  Federal  Stock  Class,  Family 
Relationship  Code  and  Local  Routing  Code  that  may  help  to 
improve  prediction  accuracy. 

B.  REGRESSION  ANALYSIS 


Regression  analysis  is  used  to  develop  a  mathematical 
model  that  describes  the  relationship  of  repair  prices  of 
inventory  items  to  other  variables.  Regression  models 
considered  in  the  thesis  research  are  linear,  possibly 
after  taking  nonlinear  transformations  of  the  response 
and/or  explanatory  variables.  Scatter  plots  are  examined 
in  order  to  better  understand  the  relationship  between 
variables.  Figure  5  shows  the  scatter  plot  of  repair 
prices  versus  replacement  prices  on  a  logarithmic  scale. 
The  relationship  between  the  two  variables  can  be 
reasonably  described  as  linear  on  a  logarithmic  scale.  The 
decision  was  therefore  made  to  examine  regression  models 
with  logarithm  transformations  applied  to  these  two 
variables . 
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Figure  5 .  Scatter  plot  of  repair  prices  versus  replacement 

prices  for  1,343  repair  items  in  the  Naval  inventory. 


The  simple  logarithmic  transformed  regression  model 
(Model  I)  is  expressed  as  follows: 


log 

V 


Y, 


X 


i  J 


A  +  A  l°g )  +  £i  r 


where  Xi  and  Yt  are  the  replacement  and  repair  prices, 
respectively,  for  inventory  item  i .  The  intercept  and  slope 
coefficients  are  jB0  and  A  >  and  £,  is  the  error  term.  It  is 
assumed  that  the  error  terms  £t  are  independent,  normally 
distributed  random  variables  with  mean  zero  and  a  common 
(but  unknown)  variance  <J~  .  Least-squares  regression  was 
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used  to  estimate  the  coefficients  of  the  regression  model, 
using  the  statistical  software  package  S-Plus@  on  data 
consisting  of  replacement  and  repair  prices  for  1,343  Naval 
inventory  items.  The  least  squares  estimates  are  shown  in 
the  following  table: 

Table  10.  Least-squares  Regression  Coefficient  for  the  Simple 
Regression  Model,  Based  on  n  =  1,343  Items. 


Coefficient 

Estimate 

Standard  Error 

t-ratio 

Intercept  ( /30  ) 

1.5858 

0 . 1571 

10.0687 

Slope  ( /?,  ) 

-0.3403 

0 .0168 

-20.2806 

Values  of  the  t-ratio  that  exceed  1.96  in  absolute  value  are 
statistically  significant  at  the  a  =  .05  test  level. 


From  Table  10,  it  is  seen  that  the  logarithm  of 
replacement  price  is  a  statistically  significant  predictor 
of  the  logarithm  of  the  ratio  of  repair  price  to 
replacement  price.  To  predict  the  repair  price  the 
following  nonlinear  transformation  is  used: 


t  =  X i  exp(/?0  +  /?,  log  AC) . 


C.  REGRESSION  ANALYSIS  WITH  CATEGORICAL  VARIABLES 

One  of  the  advantages  of  regression  models  compared  to 
ratio-estimator  models  is  the  ability  of  the  former  to 
incorporate  predictor  variables  of  diverse  types.  NAVICP 
collects  data  on  a  wide  range  of  attributes  for  the 
repairable  items  that  it  manages,  much  of  which  is 


categorical.  In  the  data  set  provided  by  NAVICP,  as  shown 
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in  Table  5,  are  attributes  such  as  Federal  Stock  Class 
(FSC)  and  Local  Routing  Code  (LRC)  that  groups  these 
inventory  items  into  their  respective  functions  and/or 
platform  types.  In  this  section  regression  models  that 
incorporate  these  categorical  attributes  as  predictors  is 
examined . 


In  the  data  provided  by  NAVICP,  restricted  to  1,343 
inventory  items  that  were  used  in  analyses,  a  total  of  101 
different  FSC  levels  and  18  different  LRC  levels  were 
represented.  For  model-building  purposes  only  those  levels 
with  twenty  or  more  inventory  items  were  considered  as 
potential  predictor  variables.  These  levels  are  shown 
below  in  Table  11. 
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Table  11.  FSC  and  LRC  levels  with  twenty  or  more  inventory 

items . 


FSC 

Description 

Number  of 
inventory 
items 

LRC 

Description 

Number  of 
inventory 
items 

5998 

Circuit  Card 

Assemblies 

240 

A$$ 

F/A18  A/D 

205 

1560 

Airframe 

Structural 

Components 

126 

c$$ 

AV-8B 

69 

2840 

Aircraft  Engines 
and  Turbines 
Components 

102 

D$  $ 

EA-6B 

64 

1615 

Helo  Rotor  Blades 

Mechanisms 

74 

E  $  $ 

F  — 14 

77 

1650 

Aircraft  Hydraulic 
Components 

72 

H$  $ 

S-3 

48 

1620 

Aircraft  Landing 
Gear  Components 

60 

L$  $ 

P-3 

74 

1680 

Miscellaneous 

Aircraft 

Components 

58 

M$  $ 

E— 2 

69 

5895 

Miscellaneous 

Communication 

Equipment 

41 

Q$$ 

Common  Systems 

78 

6625 

Electronic  Testing 
Instruments 

39 

s$$ 

Aircraft 

Engines 

130 

6130 

Non-rotating 

Electrical 

Converters 

35 

T$  $ 

Aviation 

Support 

Systems 

83 

5985 

Antennas  Related 
Equipment 

26 

u$$ 

Aviation 

Support 

Systems 

101 

4920 

Aircraft 

Maintenance 

Equipment 

22 

v$$ 

H-l/H-46 

98 

6150 

Miscellaneous 
Electrical  Power 
Equipment 

22 

x$$ 

H-3/H-56 

122 

2915 

Aircraft  Fuel 
Systems  Components 

21 

Y$  $ 

H-60 

97 

5841 

Airborne  Radar 
Equipment 

20 

LRC  is  represented  by  the  first  character  of  the  code.  For  example, 
A$$  refers  to  all  LRC  codes  that  begin  with  the  letter  "A" . 
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F-tests  for  significance  of  regression  were  performed 
to  determine  if  FSC  and  LRC  contributed  significantly  to 
the  prediction  of  log (Y./X.).  Models  II  and  III  include  FSC 
and  LRC,  respectively,  as  categorical  variables  while  Model 
IV  includes  a  combination  of  both  FSC  and  LRC  variables. 
These  models  were  fit  two  different  ways: 

Intercept-only  models .  These  models  incorporate  FSC 
and/or  LRC  as  indicator  variables,  and  the  logarithm  of 
replacement  price  as  a  numeric  predictor.  Intercept-only 
models  regression  allow  for  the  possibility  that  only  the 
intercepts  of  the  regressions  vary  with  the  categorical 
predictors . 


Intercept-and-slope  models .  These  models  incorporate 
the  same  predictor  variables  as  the  intercept-only  models, 
and  the  products  of  the  indicator  variables  with  the 
logarithm  of  replacement  price  as  predictors.  Intercept- 
and-slope  models  allow  both  the  intercepts  and  slopes  of 
the  regressions  to  vary  with  the  categorical  predictors. 


Table  12  tabulates  the  result  of  the  F-test  for 
intercept  for  Models  II,  III,  and  IV  fit  as  intercept-only 
models.  For  all  three  models,  the  null  hypothesis  is 
rejected,  thus  it  can  be  concluded  that  using  different 
intercept  terms  for  FGC  and  LRC  categories  helps  to  predict 
repair  prices. 
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Similarly  the  models  with  slope  terms  are  tested  under 
the  null  hypothesis  that  the  slope  terms  do  not  help  to 
improve  prediction  power  over  their  intercept-only 
counterparts.  As  shown  in  Table  13,  the  inclusion  of  slope 
terms  leads  to  a  statistically  significant  improvement  in 
prediction  power  for  Models  III  and  IV,  but  not  for  Model 
II . 


The  next  step  is  to  perform  stepwise  regression  to 
select  a  subset  of  variables  that  are  significant  as 
predictors.  A  commonly  used  form  of  stepwise  regression  is 
forward  inclusion.  Using  statistical  software  package 
S-Plus®’,  the  default  method  of  Efroymson' s  forward  selection 
procedure  is  employed.  This  selection  procedure  starts 
with  an  empty  subset  and  at  each  step  adds  the  independent 
variable  that  gives  the  largest  reduction  of  the  residual 
sum  of  squares.  As  each  variable  is  added  to  this  subset, 
partial  correlations  are  considered  to  see  if  any  of  the 
variables  in  the  subset  should  be  dropped  [11]  .  Table  14, 
15  and  16  summarize  the  regression  models  by  reporting 
their  coefficients,  standard  errors  and  t-ratios. 
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Table  12.  F-test  for  Categorical  Predictor  Models,  Intercepts  Only 


Model 


(FGC) 


(LRC) 


(FGC  and  LRC) 


Unexplained  variation  for  the  full  model,  SSEo 
Degrees  of  Freedom,  P0 

Unexplained  variation  for  Model  I,  SSEi 


818 . 980 


:92 . 553 


852 . 124 


!  92 . 553 


795.284 


192 . 553 


Degrees  of  Freedom,  P] 


Test  statistic  value,  F-test  = 


SSEi-SSEp 
Po  ~  Pi 


7 . 941 


4.497 


5 . 533 


l  n  ~  Po  ) 

Test  statistic  critical  value  with  significance 
level  of  0.05,  Fn,  n  =  1,343 

_ OlA)-Pi,»-Po  ' _ 

Conclusion  (a  =  .05) 


1 . 639 


1 . 699 


1.476 


Conclusion  (a  =  .05)  H0  Rejected  H0  Rejected  H0  Rejected 

The  null  hypothesis  is  that  the  additional  predictors  in  Models  II,  III,  or  IV  do  not 
improve  the  prediction  power  of  Model  I . 


Table  13.  F-test  for  Categorical  Predictor  Models,  Intercepts  and  Slopes 


Model 

Unexplained  variation  for  the  full  model,  SSEo 
Degrees  of  Freedom,  P0 

Unexplained  variation  for  the  intercept-only 
models,  SSEi 

Degrees  of  Freedom,  Pi 

(  SSE.-SSE  A 


Test  statistic  value,  F-test  = 


Po  ~  Pi 


\  n~Po  J 

Test  statistic  critical  value  with  significance 

i  i  f  n  n;  ^  i  i/ii  1.674  1.699  1.476 

level  of  0.05,  n  =  1,343 

_ ■05,p0-pi,n-p0  ' _ 

Conclusion  H0  Accepted  H0  Rejected  H0  Rejected 

The  null  hypothesis  is  that  the  slope  terms  in  Models  II,  III,  or  IV  do  not  improve  the 
prediction  power  over  the  intercept-only  counterparts. 


II 

III 

IV 

(FGC) 

(LRC) 

(FGC  and  LRC) 

805 . 843 

825.412 

766.041 

32 

30 

60 

818 . 980 

852 . 124 

795.284 

17 

16 

31 

1 . 425 

3 . 035 

1 . 689 

1 . 674 

1 . 699 

1.476 

Accepted 

H0  Rejected 

H0  Rejected 

Table  14 .  Stepwise  Regression  of  Best  Possible  Selection  for 

Predictor  using  only  intercept  terms  for  Model  II  (FSC 

grouping) . 


Coefficient 

Estimate 

Standard  Error 

t-ratio 

Intercept 

1.8154 

0.1567 

11.5842 

Log(Xi) 

-0.3700 

0.0166 

-22.2745 

FSC  Code  5998 

-0.2741 

0.0591 

-4.6355 

FSC  Code  1560 

0.3323 

0.0762 

4.3616 

FSC  Code  1615 

0.5104 

0.0960 

5.3178 

FSC  Code  1620 

0.3800 

0.1055 

3.6022 

FSC  Code  1680 

0.3039 

0.1071 

2.8378 

FSC  Code  6625 

0.3128 

0.1291 

2.4224 

FSC  Code  5841 

-0.5524 

0.1783 

-3.0978 

R2 

0.2954 

Values  of  the  t-ratio  that  exceed  1.96  in  absolute  value  are 
statistically  significant  at  the  a  =  .05  test  level  and  are  used  as 

criteria  for  adding  variables  to  the  subset  when  using  Efroymson's 
method . 

Table  15.  Stepwise  Regression  of  Best  Possible  Selection  for 
Predictor  Using  both  intercept  and  slope  terms  for 
Model  III  (LRC  grouping) . 


Coefficient 

Estimate 

Standard  Error 

t-ratio 

Intercept 

1.6890 

0.1685 

10.0215 

Log(Xi) 

0.1981 

0.0656 

3.0226 

LRC  Code  E 

-0.2754 

0.0956 

-2.8822 

LRC  Code  U 

0.2025 

0.0836 

2.4215 

LRC  Code  X 

-1.0895 

0.5503 

-1.9800 

LRC  Code  Y 

-1.3794 

0.6097 

-2.2622 

LRC  Code  A  x  Log(Xi) 

-0.3574 

0.0180 

-19.8133 

LRC  Code  Y  x  Log(Xi) 

0.1411 

0.0589 

2.3956 

R2 

0.2699 

Values  of  the  t-ratio  that  exceed  1.96  in  absolute  value  are 
statistically  significant  at  the  a  =  .05  test  level  and  are  used  as 

criteria  for  adding  variables  to  the  subset  when  using  Efroymson's 
method . 
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Table  16.  Stepwise  Regression  of  Best  Possible  Selection  for 
Predictor  using  both  intercept  and  slope  terms  for 
Model  IV  (Includes  both  FSC  and  LRC  groupings) . 


Coefficient 

Estimate 

Standard  Error 

t-ratio 

Intercept 

1.6598 

0.1567 

10.5913 

Log(Xi) 

0.0371 

0.0092 

4.0498 

FSC  Code  1560 

0.3307 

0.0759 

4.3556 

LRC  Code  E 

-0.2728 

0.0928 

-2.9407 

LRC  Code  U 

0.2572 

0.0820 

3.1343 

FSC  Code  5998  x  Log(Xi) 

-0.3562 

0.0167 

-21.3800 

FSC  Code  1560  x  Log(Xi) 

-0.0317 

0.0065 

-4.8461 

FSC  Code  1650  x  Log(Xi) 

0.0430 

0.0101 

4.2616 

FSC  Code  1680  x  Log(X±) 

0.0461 

0.0110 

4.2134 

FSC  Code  5895  x  Log(Xi) 

0.0335 

0.0115 

2.9208 

LRC  Code  A  x  Log(Xi) 

-0.0528 

0.0174 

-3.0336 

LRC  Code  U  x  Log(X±) 

0.0214 

0.0108 

1.9815 

R2 

0.3146 

Values  of  the  t-ratio  that  exceed  1.96  in  absolute  value  are 
statistically  significant  at  the  a  =  .05  test  level  and  are  used  as 

criteria  for  adding  variables  to  the  subset  when  using  Efroymson's 
method . 


To  predict  the  repair  price  the  following 
transformation  is  used: 


For  Model  II (FSC  Code  with 
Yt  =  Xt  exp  (A + A  log  xi+  A2A998 ,/  +  A  A  560, 


intercept  terms  only) : 

+  AA615 +  A5  A  620,/  +  As  A  680,/  +  A  Ae25,/  +  AA841 ,/  ) 


where,  for  example, 

J 1,  item  i  has  FSC  code  5998 
5998,7  [o,  otherwise 

and  other  FSC  codes  are  represented  similarly. 

•  For  Model  III  (LRC  Code  with  both  intercept  and  slope 
terms) : 


Y;  =  X,- 


exp  ( Ao  +  A  log  A  +  PiLE,i  +  A  A/,/  +  A  Ar ,/  +  AsAy  +  As ^aj  1°§  A'  +  l°g  A' ) 


41 


where,  for  example 


Jl,  item  i  has  LRC  code  beginning  with  "A" 
Aj  [0,  otherwise 

and  other  LRC  codes  are  represented  similarly. 


•  Model  IV  (FSC  and  LRC  Code  with  both  intercept  and  slope 
terms) 


=  ^  exp  (p0  +  A  log  A,  +  A/7,560,  +  hLE,i 
A^1650,i  log  A,.  +  MfiSO,-  !°g  X,  +  A/75895, 


+  Pa^U,i  +  ft 5 ^5998,1  +  ft 6 ^1560, i  ^°S 

log  X,  +  log  JIT,  +  A,t,  logw) 


D.  RESULTS  OF  ANALYSIS 

Figure  6  shows  box  plots  of  the  relative  prediction 
errors  for  each  of  the  four  regression  models  that  were 
considered.  Again,  Model  I  use  only  log(A)  (the  logarithm 
of  replacement  price)  as  a  predictor  variable  for  log(T  /  A) 
(the  logarithm  of  the  ratio  of  repair  price  to  replacement 
price)  .  Model  II  includes  FSC  categories  into  the  model, 
Model  III  includes  LRC  categories  into  the  model,  and  Model 
IV  includes  both  FSC  and  LRC  categories  into  the  model. 
Relative  prediction  error  is  given  by  the  following: 


R,  = 


Z-Z 


Z 


where 


K  =  Afexp  (L.), 

and  Lt  is  the  fitted  value  of  log(l^/A;.). 
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The  box  plots  indicate  that  the  spread  in  relative 
prediction  errors  increases  moderately  with  the  replacement 
price;  i.e.,  more  expensive  items  have  larger  prediction 
errors  not  only  in  an  absolute  sense,  but  also  as  a 
percentage  of  the  replacement  price.  However,  relative 
errors  in  the  highest  price  category  are  reduced  when 
compared  to  the  modified  ratio  method  as  described  in 
Chapter  III.  Model  I,  II  and  IV  displayed  rather  similar 
results  in  terms  of  relative  prediction  error  in  the 
highest  price  category.  Regression  with  the  inclusion  of 
categorical  variables  shows  slight  improvement  in  relative 
prediction  error. 


Table  17  also  indicates  that  for  75  percent 
observations,  the  relative  prediction  error  ranges 
percent  to  96  percent. 


of  the 
from  89 
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Table  17.  Quantiles  of  Absolute  Relative  Prediction  Error  (Percentages)  for  Four  Regression 

Models  Considered. 


Category 

Sample 

Size 

Model  I 

Model  II 

2  5th 

Percentile 

Median 

7  5th 

Percentile 

Mean 

2  5th 

Percentile 

Median 

7  5th 

Percentile 

Mean 

up  to  $999 

33 

18.05 

28.39 

44.80 

39.70 

21.76 

28.98 

56.54 

49.44 

$1,000  - 
$2, 999 

188 

19.23 

37.75 

60.45 

59.23 

18.93 

37.41 

59.03 

57.18 

$3,000  - 
$  9 ,  999 

457 

24.61 

45.38 

69.28 

71.19 

22.37 

43.92 

66.20 

67.55 

$10,000  - 
$24, 999 

354 

26.68 

54.05 

79.23 

94.11 

26.21 

49.93 

73.41 

86.23 

$25,999  - 
$49, 999 

155 

31.45 

53.22 

84.99 

121.40 

26.81 

52.43 

85.52 

114.11 

$50, 000 
and  above 

156 

31.20 

58.41 

95.84 

148.67 

28.53 

60.46 

95.28 

137.95 

Category 

Sample 

Size 

Model  III 

Model  IV 

2  5th 

Percentile 

Median 

7  5th 

Percentile 

Mean 

2  5th 

Percentile 

Median 

7  5th 

Percentile 

Mean 

up  to  $999 

33 

19.16 

28.83 

49.98 

40.65 

22.54 

29.68 

54.18 

47.23 

$1,000  - 
$2, 999 

188 

19.81 

36.61 

56.99 

58.57 

18.55 

36.12 

58.50 

57.94 

$3,000  - 
$  9 ,  999 

457 

21.82 

43.39 

68.31 

68.31 

22.23 

42.11 

64.49 

65.95 

$10,000  - 
$24, 999 

354 

22.65 

51.17 

76.41 

87.26 

22.80 

46.58 

73.03 

80.99 

$25,999  - 
$49, 999 

155 

31.08 

52.64 

83.27 

117.85 

27.59 

52.15 

92.54 

114.79 

$50, 000 
and  above 

156 

28.73 

58.87 

92.58 

144.00 

26.37 

57.19 

88.90 

140.44 

Table  18  summarizes  the  results  of  the  regression 
analysis.  It  can  be  observed  that  there  is  improvement 
with  the  inclusion  of  categorical  variables,  especially 
with  FSC.  From  both  Table  17  and  Table  18,  it  is  observed 
that  Model  IV  yield  the  best  predictive  power  with  highest 
R2  and  lowest  prediction  error  among  all  four  models. 


Table  18 .  Summary  of  Regression  Analysis 


Model 

Equation 

R2 

I 

Yt  =  X(.  exp(/?0  +  A  log  X(.) 

0.2347 

II 

Yi  =  V,  exp(p0  +  A  log  X,.  +  A^5998,,-  +  AF1560 ,i 
“*~A^1615,i  PsF\620,i  “*“A-^1680,i  A  ^6625,1  A^5841,i  ) 

0.2954 

III 

Yi=xi  exp(A  +  A  log Xi+^2L,i+^LUi 

+  ^Lx.+p5LY.+p6LA'i  log  Xi+p1Ly.  log  x) 

0.2699 

IV 

Yt  =  X;  exp  (A  +  A  l°gX  +  Mm,  +  A^ev  +  Pa^uj 
+  p5F5<m,i  l°gX;  +  P6Fl560i  log X,  +  A Fl650  ,  log X,  +PHFm0i  logX, 

+  A  ^5895,,  logX,.  +  Ao  LA,i  logX,.  +  AiV,  logX,.) 

0.3146 

However  the  most  substantial  improvement  is  found  by 
comparing  Model  II  to  Model  I.  The  percentage  of  variance 
explained  in  the  response  variable  increases  from  23.5 
percent  to  29.5  percent  with  the  addition  of  FSC  levels  as 
predictor  variables.  The  practical  effect  of  this 
improvement  can  be  seen  with  the  240  items  in  the  sample 
that  belong  to  the  FSC  category  "5998".  For  these  items, 
the  median  (50th  percentile)  and  upper  quartile  (75th 
percentile)  of  absolute  relative  errors  were  42.8  percent 
and  70.5  percent  for  Model  II,  compared  to  47.3  percent  and 
114.6  percent  respectively  for  Model  I. 
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Figure  7  shows  a  scatter  plot  of  the  residuals  against 
the  predicted  values  for  Model  II.  Ideally,  this  plot  does 
not  reveal  a  nonlinear  pattern  or  evidence  of  unequal 
variances  (heteroscedast icity ) .  The  scatter  plot  does 
reveal  some  evidence  that  the  variance  of  the  residuals 
decreases  with  the  size  of  the  predicted  value.  This 
pattern  suggests  that  greater  efficiency  may  be  realized  by 
using  weighted  least-squares  regression,  although  ordinary 
least  squares  also  provides  unbiased  estimates  of  the 
regression  coefficients.  The  normal  quantile-quantile  (QQ) 
plot  of  the  residuals,  shown  in  Figure  8,  indicates  that 
the  distribution  of  residuals  deviates  from  normality 
mainly  due  to  skewness  in  the  left-hand  tail.  Despite 
these  departures  from  ideal  model  assumptions.  Model  II 
appears  to  be  a  good  fit  for  this  analysis,  and  an 
improvement  over  the  methodology  currently  in  use. 

The  next  section  will  consider  how  a  change  in  the 
prediction  methodology  would  have  affected  assessment  of 
the  FIRST  program. 
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Figure  8.  Quantile-normal  Plot  of  Residuals  from  Model  II 


Quantiles  of  Standard  Normal 


E.  VALIDATING  THE  ANALYSIS  WITH  DATA  FROM  THE  FIRST 

PROGRAM 

For  the  FIRST  Program,  BCA  analysis  was  performed  by 
NAVICP  to  estimate  the  repair  prices  of  its  inventory 
items.  Twenty  items  were  reviewed  in  which  all  belongs  to 
the  last  three  replacement  price  categories  (Table  19)  .  It 
is  observed  that  current  NAVICP  prediction  methodology  has 
the  tendency  to  overestimate  repair  prices  especially  in 
this  category,  whereby  these  prediction  errors  had  the 
greatest  fiscal  impact.  Details  of  the  estimated  repair 
prices  using  NAVICP' s  BCA  analysis  are  shown  in  Table  20. 
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DoD  IG  report  claimed  that  NAVICP's  BCA  repair  prices 
overstated  the  actual  repair  price  [1]. 


Table  19.  Category  listings  and  sample  size  of  the  repair  items 

reviewed  by  DoD  IG 


Category 

Sample  Size 

$10,  000  -  $24,  999 

3 

$25, 999  -  $49, 999 

5 

$50,000  and  above 

12 

Table  21  shows  the  analysis  performed  using  the 
current,  modified  and  regression  method  as  discussed  in 
previous  section  to  determine  the  predictive  power  of  the 
respective  models.  Comparing  current  prediction  method 
adopted  by  NAVICP  ($65.5  million)  to  the  proposed 
regression  method  ($35.8  million),  the  overstatement  of 
benefit  from  the  PBL  that  DoD  IG  claims  would  have  been 
reduced  by  $29.7  million.  This  shows  that  the 
relationship  of  the  repair  prices  to  replacement  prices  is 
indeed  non-linear.  Hence  using  the  regression  method  would 
reduce  the  prediction  error  of  repair  prices  by  almost  $30 
million . 
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Table  20.  Analysis  of  NAVICP  BCA  repair  prices  (FIRST  Program) 


FSC 

NUN 

5  Yr 

Demand 

Repair 

Prices 

Total  Cost 

1430 

014553659 

164 

$13,303 

$2, 181, 651 

1620 

014636970 

109 

$8, 115 

$884,573 

1620 

014668717 

77 

$41, 987 

$3, 232, 972 

1630 

014551442 

176 

$7, 970 

$1, 402, 794 

1650 

014552590 

123 

$9, 130 

$1, 123, 039 

1650 

014553668 

200 

$7,704 

$1,540, 846 

1650 

014554490 

514 

$20, 850 

$10,716, 900 

1650 

014691468 

217 

$15,339 

$3, 328,498 

1680 

014552537 

469 

$5, 001 

$2,345, 375 

1680 

014774914 

362 

$55, 121 

$19, 953, 784 

1680 

014782049 

105 

$8, 722 

$915, 810 

1680 

014800498 

59 

$38, 632 

$2,279,289 

1720 

014551420 

219 

$3, 990 

$873, 786 

2520 

014726137 

66 

$28,763 

$1, 898,325 

4320 

014545082 

39 

$14, 664 

$571, 896 

4320 

014552588 

199 

$7, 834 

$1,558, 892 

5998 

012960824 

141 

$3, 622 

$510, 646 

5998 

014658626 

532 

$5, 118 

$2,723, 031 

6115 

014553692 

104 

$38, 484 

$4,002,336 

6615 

014820902 

157 

$22, 606 

$3,549, 079 

Table  21.  Comparison  of  Predicted  Repair  Prices  between  Current, 

Modified  and  Regression  Method  (Model  II) . 


Prediction  Method 

Total  Predicted 
Repair  Price 

Total  Actual 
Repair  Price 

Difference 

Percent 

Prediction  using  NAVICP  BCA 
in  IGDOD  Report 

$65.6  million 

$21.4  million 

$44.2  million 

206 

Prediction  using  Current 
Prediction  Method 

$68.9  million 

$21.4  million 

$47.5  million 

222 

Prediction  using  Modified 
Ratio  Method 

$55.8  million 

$21.4  million 

$34.4  million 

161 

Prediction  using  Regression 
Method  (Model  II) 

$35.9  million 

$21.4  million 

$14.5  million 

67 

Note:  Analysis  is  based  on  the  20  items  identified  in 
DoD  IG  report  [1]  . 
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F. 


VALIDATING  ANALYSES  WITH  NEWLY  PROVISIONED  ITEMS  FOR 
FY  2002 


In  addition  to  validating  the  regression  methodology 
with  the  repair-price  data  from  the  FIRST  program,  an 
analysis  was  also  conducted  with  newly-provisioned  items 
for  FY  2002.  After  using  the  filtering  techniques 
discussed  in  Chapter  III,  there  were  a  total  of  575  newly 
provisioned  inventory  items  for  FY  2002  that  were  available 
for  analysis . 


Table  22  summarizes  absolute  relative  prediction 
errors  for  the  current,  modified  and  regression  prediction 
(Model  II)  methodologies.  There  is  no  clear  pattern  that 
suggests  that  one  methodology  outperforms  the  others  across 
all  categories  of  items.  It  is  of  interest  to  note  that, 
in  the  highest-cost  category,  the  regression  method 
produces  the  largest  median  of  absolute  relative  prediction 
errors,  but  also  the  smallest  75th  percentile  of  these  same 
quantities . 
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Table  22.  Quantiles  of  Absolute  Relative  Prediction  Errors  (Percentages)  Using  the  Current, 
Modified  and  Regression  Method  (Model  II)  for  Newly  Provisioned  FY  2002  Inventory 

Items . 


Category 

Sample 

Size 

Current  Ratio  Method 

Modified  Ratio  Method 

Regression  Method  (Model  II) 

25,n 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

up  to 
$999 

32 

26.91 

48.34 

86.10 

64.15 

23.66 

44.54 

82.61 

61.06 

32.20 

46.58 

87.89 

98.18 

$1,000  - 
$2, 999 

71 

26.60 

48.69 

79.34 

90.23 

24.86 

49.71 

79.75 

87.99 

23.07 

40.63 

76.15 

62.95 

$3,000  - 
$9, 999 

173 

17.02 

35.77 

78.77 

82.39 

18.07 

37.19 

73.55 

78.14 

20.28 

42.88 

72.63 

63.03 

$10,000 

$24, 999 

128 

18.88 

44.95 

73.71 

75.95 

19.63 

44.83 

72.10 

75.04 

26.50 

50.30 

77.37 

80.90 

$25, 999 

$49, 999 

54 

40.49 

61.81 

116.64 

104.68 

39.83 

62.75 

119.02 

105.87 

22.62 

49.09 

72.89 

68.29 

$50,000 

and 

above 

117 

6.69 

42.11 

89.90 

162.42 

17.35 

35.98 

72.06 

128.51 

22.75 

42.19 

72.77 

68.76 

THIS  PAGE  INTENTIONALLY  LEFT  BLANK 
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V. 


CONCLUSIONG  AND  RECOMMENDATIONS 


A.  INTRODUCTION 


The  intention  of  this  research  is  to  develop  a 
predictive  model  for  estimation  of  repair  prices.  This 
research  was  initiated  after  DoD  IG  instigated  a  review  of 
the  FIRST  Program  awarded  to  Boeing  whereby  the  Navy 
largely  underestimated  the  repair  prices.  It  was  observed 
that  of  the  20  inventory  items  reviewed  by  DoD  IG,  twelve 
are  from  the  highest  price  category  as  shown  in  Table  19. 

B.  CONCLUSION  AND  RECOMMENDATIONS 


In  this  research,  three  methodologies  were  considered. 
The  first,  a  modified  ratio  methodology,  is  somewhat 
similar  to  the  current  ratio  methodology  used  by  NAVICP. 
The  second  is  based  on  a  regression  with  logarithmic 
transformations  of  both  the  response  and  predictor 
variables,  and  the  third  is  an  extension  which  includes 
categorical  variables  in  regression  analyses. 

It  is  observed  that  the  largest  share  of  the  overall 
prediction  error  occurs  with  items  having  the  largest 
replacement  prices  ($10,000  and  above) .  But,  in  the 
highest  cost  category,  there  is  noticeable  improvement  if 
the  modified  ratio  methodology  is  used.  Using  the  modified 
ratio  methodology,  the  difference  between  predicted  and 
actual  repair  prices  was  reduced  to  approximately  $34 
million.  However,  this  error  is  still  large  in  light  of 
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the  fact  that  the  actual  repair  price  of  the  items 
considered  was  only  $21  million.  This  finding  suggests 
that  there  may  have  been  much  to  be  gained  by  examining 
alternative  prediction  methodologies. 


One  suggestion  highlighted  in  this  research  is  the  use 
of  transformed  regression  analysis.  As  shown  in  Figure  5, 
the  relationship  of  replacement  to  repair  price  is  fairly 
well  correlated  on  a  logarithmic  scale.  Additional 
analysis  using  categorical  variable  are  also  presented  in 
Chapter  IV. 


Box  plots  for  the  regression  methods,  as  shown  in 
Figure  6,  all  indicates  improvement  in  the  higher  price 
category,  especially  model  IV  (i.e.  regression  analysis 
with  both  FSC  and  LRC  as  categorical  variable)  which  yield 
the  best  results.  However  it  can  be  noted  that  all 
regression  methodology  though  performed  fairly  well,  had 
hardly  any  vast  distinctive  improvement  among  each  other, 
even  with  the  inclusion  of  categorical  variables.  Thus  it 
seems  that  analysis  with  regression  methodology  should 
suffice  for  the  estimation  of  repair  prices. 


Similarly,  the  predictive  ability  of  the  regression 
method  is  tested  with  the  repair  items  reviewed  by  DoD  IG 
[1] .  This  model  presented  significant  improvement  in  the 
predictive  ability  and  the  prediction  error  drops  from  an 
observed  $47  million  (using  current  ratio  method)  to  about 
$14  million. 
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It  is  recommended  that  NAVICP  consider  the  use  of 
regression  analysis  with  logarithmic  transformations  to 
develop  it  repair  matrix.  These  models  are  able  to 
incorporate  predictors  other  than  the  replacement  price, 
including  FSC  and  LRC,  which  were  found  to  be  of  some 
value.  With  a  concerted  research  effort  it  is  possible 
that  other  effective  predictors  may  also  be  found. 
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APPENDIX  A:  LISTING  OF  FSC  AND  LRC  GROUPING  WITH 

DESCRIPTION 


FSC  Code  and  Description 

1210-1290  Fire  Control  Equipment 
1305-1398  Ammunition  and  Explosives 
1410-1450  Guided  Missiles 

1560-1560  Aircraft  Airframe  Structural  Components 
1610-1615  Aircraft  Propellers  and  Helo  Rotor  Blades  & 
Drive  Mechanisms 

1620-1680  Aircraft  Components  and  Accessories 
1710-1990  Aircraft  Launch  Land  &  Ground  Service  Equip 
2010-2090  Ship  &  Marine  Equipment 
2210-2350  Railway  Equipment 
2410-2430  Tractors 

2510-2590  Vehicular  Equipment  Components 
2610-2640  Tires  and  Tubes 
2805-2895  Engines  Turbines  &  Components 
2910-2995  Engine  Accessories 

3010-3040  Mechanical  Power  Transmission  Equipment 
3110-3130  Bearings 

3210-3230  Woodworking  Machinery  &  Equipment 
3405-3470  Metalworking  Machinery 
3510-3590  Service  And  Trade  Equipment 
3605-3695  Special  Industry  Machinery 
3710-3770  Agricultural  Machinery  &  Equipment 
3805-3895  Construction,  Mining,  &  Excavating 
3910-3990  Materials  Handling  Equipment 
4010-4030  Rope,  Cable,  Chain,  &  Fittings 
4110-4140  Refrigeration  &  Air  Conditioning 
4210-4240  Fire  Fighting,  Rescue  &  Safety  Equipment 
4310-4330  Pumps  and  Compressors 

4410-4470  Furnace,  Steam  Plant  &  Drying  Equipment 
4510-4540  Plumbing,  Heating  &  Sanitation  Equipment 
4610-4630  Water  Purification/Sewage  Treatment  Equipment 
4710-4730  Pipe,  Tubing,  Hose,  &  Fittings 
4810-4820  Valves 

4910-4960  Maintenance  &  Repair  Shop  Equipment 
5110-5180  Hand  Tools 
5210-5280  Measuring  Tools 
5305-5365  Hardware  and  Abrasives 

5410-5450  Prefabricated  Structures  &  Scaffolding 
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FSC  Code  and  Description 

5510-5530  Lumber,  Millwork,  Plywood  &  Veneer 

5610-5680  Construction  &  Building  Materials 

5805-5895  Communication  &  Detection  Equipment 

5905-5999  Electrical  &  Electronic  Equipment  Components 

6010-6080  Fiber  Optics  Materials  &  Components 

6105-6150  Electric  Wire,  Power  &  Distribution  Equipment 

6210-6260  Lighting  Fixtures  and  Lamps 

6310-6350  Alarm,  Signal  &  Security  Detection  Systems 

6505-6550  Medical,  Dental  Equipment  &  Supplies 

6605-6695  Instruments  &  Laboratory  Equipment 

6710-6780  Photographic  Equipment 

6810-6850  Chemicals  &  Chemical  Products 

6910-6940  Training  Aids  &  Devices 

7010-7050  Data  Process  &  Support  Equipment 

7105-7195  Furniture 

7210-7290  Household,  Commercial  Furnishings&  Appliances 
7310-7360  Food  Preparation  &  Serving  Equipment 
7420-7490  Office  Machines,  Text  Proc  Equip  &  Visible 
Record  Equipment 

7510-7540  Office  Supplies  &  Devices 

7610-7690  Books,  Maps  &  Other  Publications 

7710-7740  Musical,  Phonographs  &  Home-Type  Radios 

7810-7830  Recreational  &  Athletic  Equipment 

7910-7930  Cleaning  Equipment  &  Supplies 

8010-8040  Brushes,  Paints,  Sealers  &  Adhesives 

8105-8145  Containers,  Packaging,  &  Packing  Supplies 

8305-8345  Textiles,  Apparel,  Shoe  Findings-Tent s&Flags 

8405-8475  Clothing,  Individual  Equipment  &  Insignia 

8510-8540  Toiletries 

8710-8730  Agricultural  Supplies 

8810-8820  Live  Animals 

8900-8999  Subsistence-Food 

9110-9160  Fuels,  Lubricants,  Oils  &  Waxes 

9310-9390  Nonmetallic  Fabricated  Materials 

9410-9450  Nonmetallic  Crude  Materials 


9505-9545  Metal  Bars,  Sheets,  and  Shapes 
9610-9680  Ores,  Minerals  &  Their  Primary  Products 
9905-9999  Miscellaneous 
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LRC  Code  and  Platform 

A  F/A-18  A/D 

B  F/A-18  E/F 

C  AV-8B 
D  EA-6B 
E  F-14 
G  V-22 
H  S-3 

J  C-130,E-6,F-5,F-16, T-38 
L  P-3 
M  E-2 
P  C-2 

Q  Common  Systems 
R  T-45 

S  Aircraft  Engines 

T/U  Aviation  Support  Systems 

V  H-l/H-46 
X  H-3/H-53 

Y  H-60 
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